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Abstract 

We propose a new method of the construction of the asymptot¬ 
ically efficient estimator-processes asymptotically equivalent to the 
MLE and the same time much more easy to calculate. We suppose 
that the observed process is ergodic diffusion and that there is a learn¬ 
ing time interval of the length negligeable with respect to the whole 
time of observations. The preliminary estimator obtained after the 
learning time is then used in the construction of one-step and two- 
step MLE processes. We discuss the possibility of the applications 
of the proposed estimation procedure to several other observations 
models. 
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1 Introduction 

We consider the problem of parameter estimation by the continuous time 
observations X'^ = (W,0 < t < T) of the diffusion process 

dXt = S {d, Xt) dt + a (W) dWt, Xo, 0<t<T. 
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We suppose that the process (W)t>o has ergodic properties with the den¬ 
sity of invariant distribution /(d, x). The functions S{'d,x) and cr (x) are 
known, smooth and the parameter -d G 0 C It is known that the 

maximum likelihood estimator (MLE) dx constructed by the observations 
X'^ (under regularity conditions) is consistent, asymptotically normal and 
asymptotically efficient (see, e.g., m or m)- 

We consider here slightly different statement of the problem. Suppose 
that we are interested by an estimator-process m 0 < f < T), where 

depends on the observations X* = (X^, 0 < s < f) only. The need of such 
on-line estimators naturally arises in many problems, for example in adaptive 
control. We used such estimators in the construction of the approximation 
of the solution of backward stochastic differential equation [ 15 ], [ 12 ], where 
the estimator-processes were of the one-step MLE-type. Note that there is 
a large literature on stochastic approximation, which provides satisfactory 
solutions (see, e.g.; [9], and references therein). For continuous time systems 
such problems were studied for example in [2] and [21] . In the last work there 
was proposed a recurrent asymptotically efficient estimation in the case of 
observations 


dXt = S{'d,t)dt + a{t)dWt, 0<t<T. 

Another recurrent estimator-process for diffusion processes was studied in 
|18j . We have to mention that the estimator-process proposed in our work is 
not recurrent in this sense. The right-hand side of the equation for it does 
not depend on the preceding values of this estimator (see ([T|) below). 

For diffusion processes observed in discrete times the adaptive estimation 
of the parameters of the trend and diffusion coefficients were studied in many 
works, see, e.g., m, 0. 121.1231 and the references therein. Note that 
in the works 1221 and [23] the proposed multi-step adaptive procedures of 
parameter estimation allows to improve the initial bad rates of convergence 
up to asymptotically efficient {good) rates. 

The studied in the present work estimator-processes are based on the 
one-step MLE structure. Recall that the one-step MLE was introduced by 
Le Cam [16] in 1956. The dehnition and properties of it in i.i.d. case can 
be found, for example, in im. Let us remind it’s construction. Suppose 
that the observed i.i.d. random variables X"’ = (Xi,... ,X„) have smooth 
density function / {d, x) with unknown parameter -d G 0 C 71^ and we 
have to estimate d. Suppose as well that we are given an estimator dn 
which is consistent and asymptotically normal y/n [dn — do) ^ W (0, D (’do)) 
with a bad d x d limit covariance matrix ©(do) > I('do)~^, i.e., the matrix 
D ('do) —I ('*^o)~^ is positive dehnite. We say “bad” because D {do) is not equal 
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to the the inverse Fisher information matrix I ("i^o) ^, which is limit covariance 
of asymptotically efficient estimators. The one-step MLE is dehned as follows 

1 ?; = 4 + n-'-i (^„)"‘ V 

Here and in the sequel dot means derivation w.r.t. d. The estimator has 
already the good limit covariance matrix 

Therefore this procedure allows us to improve any estimator with good rate 
^/n but bad limit covariance up to asymptotically efficient. 

The one-step MLE for ergodic diffusion process can be dehned by a 
similar way. Suppose that we have a preliminary estimator (say, mini¬ 
mum distance estimator or estimator of the method of moments), which is 
consistent and asymptotically normal : 

Vf{^T-'d) ^ . 

The limit variance D (f?) > I('d)~^, where I (’d) is the Fisher information 
matrix 







/(■!?, x) dx. 


Here and in the sequel A* means transposition of A. 

Following the same “one-step” idea we can improve this estimator up to 
asymptotically efficient as follows 


id 


★ _ 

T — 





[dXt-S{dT,x) df] . 


( 1 ) 


whehe id1^ is the one-step MLE. Of course, the special attention have to 
be paid for the dehnition of the stochastic integral because the estimator 
idr depends on the whole observations This estimator is consistent, 

asymptotically normal 


and asymptotically efficient (see, e.g., m)- Note that the preliminary esti¬ 
mator here has a good \/T rate of convergence. 
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Recently Kamatani and Uchida |1] considered the problem of parameter 
estimation by the discrete time observations X'^ = = ihn, 0 < i < n) 

of ergo die diffnsion process 

dX* = 5 (di, Xi) dt + a (^?2, X^) dfXi, Xo, 0 < f < T. 

Here d = ('di,'d 2 ) is unknown parameter. They proposed a modiheation of 
Newton-Raphson (N-R) procedure, with the initial estimators of bad (non- 
optimal) rate of convergence and showed that the multi-step N-R procedure 
allows to obtain the asymptotically efficient estimators with the good rates. 
The asymptotic is —)■ 0 and nhn —t cxo. 

In our work we consider a similar construction but based on the modihea¬ 
tion of one-step MLE procedure. We propose estimator-processes < 

t < T and 'd**rp,T^ ^ t ^ T, where h < 1 and andhave one-step 
MLE-type structure. As preliminary estimator •drps we take an estimator con¬ 
structed by the hrst observations X^'* on the time interval [O, T^~\. Therefore 
the preliminary estimator has a bad rate of convergence due to the length of 
the learning interval. 

Then we propose one-step (for 6 G (|, l)) and two-step (for 6 G (|, |]) 
MLE-processes. For example, the one-step MLE-process is 

= drs + f [dXt - 5 , x) dt] , 

’ Jt^ a (x) 

where < t < T. It is shown that this estimator-process is consistent, 
asymptotically normal and asymptotically efficient. Note that the calculation 
of this estimator-process is much more simple than the calculation of the MLE 
dt,T for all f G [T^T]. 

2 Auxiliary results 

We are given a probability space {f2, X, P} with hltration (Xt)^>g satisfying 
the usual conditions and the Wiener process W = (IW,Xt,f >0). Suppose 
that for all -d G 0 C (0 is an open bounded set) the stochastic process 
X = (Xi,Xi,f > 0) satishes the stochastic differential equation 

dXi = .S (d, Xt) dt + a {Xt) dlW, Xo, t > 0, (2) 

where (Xq, Xq) is the initial value. The functions S (d, x) and a (x)^ > 0 are 
such that this equation has a unique strong solution on any hxed interval 
[0, T] and that the measures < d G 0 [ induced in the measurable space 


4 



(C [0,T] ,^t) of its realizations are equivalent (see conditions for example 
here HI). Moreover we suppose that the process Xt,t > 0 has ergodic 
properties with the density of invariant distribution 




1 

-o exp 

G (’d) cr (x) 


\ Jo cr (y) 





where G (d) is the normalizing constant [7] . The random variable with such 
density we denote as ^ and suppose that Xq has the same density function. 
This condition makes the stochastic process stationary. 

The sufficient condition for the existence of ergodic properties we take as 
in HH Dehne the class of functions 


P = {/i(-): |M^)I < C'(l + 


where the constants (7 > 0, g > 0 do not depend on d in the case of the func¬ 
tion S (d, x) and its derivatives and can be different for different functions. 
Condition Ao (0). The functions S (d, •), a (■)^^ G V and 


lim sup 


sgn (x) S (d, x) 
a (x)^ 


< 0 . 


The smoothness condition: the function S (d, x) has two continuous partial 
derivatives w.r.t. d and these derivatives belong to V. 

These derivatives we denote as : S (d, x) (vector) and S (d, x) {d x d 
matrix). 

The identifiability condition: for any z/ > 0 


inf inf E^„ 


(S{t)Ao)-S{t}oAo) 

V ^ (^o) 


2 


> 0 . 


Here the r.v. has the density function / (do,x). 

The Fisher information matrix I (d) is uniformly non degenerate (below 
A e R^) 


inf inf A*I (d) A > 0. 
i?ee|A|=i 

The set of all these conditions we call Regularity conditions. 

We have to estimate d by the observations X^ = (Wj 0 < s < f) for all 
t G (0,T] and to describe the properties of the estimator-process (d (f) 

0 < t < T), where d (f) = d {t, X^). We would like to obtain an estimator- 
process which has asymptotically optimal in some sense properties. 
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It will be convenient to change the variables t = tT, t G (0,1] and to 
stndy the random processes 'dr,T,0 < t < 1, where For 

simplicity of exposition we will write as dr- 

Introdnce the likelihood ratio-process V G 0, 0 < r < 1, 

where 


= exp 


r^ s{^,x,) 

Jo a {Xsf 


dXs- 


0 2a{X,f j 


Note that for any r G (0,1] the family of measures G ©j is 

locally asymptotically normal (LAN) in 0, i.e.; the likelihood ratio-process 


ZtT iu) = 




0 < r < 1, 


with u such that -do + G 0, admits the representation 


Zr (u) = exp <1 u*A-r (do, - -jU*Ir (do) U + Tt 


Here (do) = tI (do), t 0 and the score-function (vector-process) 


A. (do,X"^) 


^ S{^0,X,) 

VT Jo a {Xsf 


[dA, 


F(do,A,)ds] ^AA(0,I. (do)). 


Therefore we have the following Hajek-Le Cam-type low bound for poly¬ 
nomial loss function (p > 0): for all estimator-processes dT-,0 < r < 1 and 
all do G 0 and r G (0,1] 


lim lim sup 

>-0 T—>-oo ji?—i^ol <1^ 


E,< 


^/^I, (do)'/' (d. - d) 


p 


>Eicr, 


( 3 ) 


where the vector C ~ A/'(0, J), J is unit d x d matrix (see, e.g., [5] or m)- 
Therefore the estimator process dr, 0 < r < 1 we call asymptotically 
efficient if for all do G 0 and all r G (0,1] we have the equality 


lim lim 

5—>0 T—>-00 


sup 

li?—'i?o|<(5 


E,< 


^/TIr (do)'/' (dr - d) 


P 


Eicr. 


( 4 ) 


One solution of this problem is to introduce the MLE process dr, 0 < r < 1 
dehned by the equation 

H (dr, x^^) = sup H (d, X"^) , r G (0,1]. (5) 

^ ^ i?e0 
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It is known that the estimators r G (0,1] nnder regnlarity conditions 
are consistent and asymptotically normal ("do is the trne valne) 


Vf^dr-'&o) 

Moreover we have the nniform on i!) convergence of moments 


lim 

T^OO 


^p/ 2 e 


I. (^) 


1/2 



Eicr. 


Therefore the MLE-process 19^0 < r < 1 is asymptotically efficient (see 

d]). 

Unfortnnately except the linear case S (d, x) = d*h [x) the calcnlation for 
all T G (0,1] of the MLE-process as solntion of the equation (jH) is computa¬ 
tionally very difficult problem and we have to seek another estimator-process 
which is computantionally much more simple. The goal of this work is to 
describe such class of estimator-processes. The construction of the proposed 
estimator-processes is based on the development of the well-known one-step 
MLE device. 


3 Main result 

We consider the problem of estimation i? by observations X* for t G [T^, T]. 
The corresponding estimators we study after the change of variables t = 
tT. Therefore we are interested by the construction of the estimator-process 
1 ?*^, r G [r^, 1], where ts = We show that if h G (|, 1), then the one- 

step MLE-process is asymptotically normal and asymptotically efficient. If 
S G (|, |], then we propose two-step MLE-process with the same asymptotic 
properties. 

3.1 One-step MLE {6 e (^, 1)) 

Introduce the learning interval 0 < f < T^, where 6 G (|, 1) and denote 
by an estimator of parameter which is uniformly on compacts A C 0 
asymptotically normal 

r^(4-^o)^AA(0,D(4)), (6) 

where ts = —)■ 0 and the matrix D {'do) of limit covariance is bounded. 

Moreover we suppose that we have the convergence of all polynomial mo¬ 
ments too: for all p > 0 

supT^E^J^?,,-4r<C', (7) 

)9o6A 
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where the constant C > 0 does not depend on T. The regnlarity condi¬ 
tions providing these properties of the MLE, minimnm distance estimators 
(MDE), bayesian estimators (BE) and the estimators of the method of mo¬ 
ments (EMM) can be fonnd, for example, in [TT], Chapter 2. Therefore as 
preliminary estimator we can take one of them. 

The one-step MLE-process we constrnct as follows: 

c = Br. + A, (4 , A-fl), T e In, 1 ] (8) 


where 


A, (tf, .YJI) = ^ r [dx, - S (d, A-,) di|. (9) 

V tT Jts a {Xt) 

Introdnce the random process 

Vt,t i^o) = t^/TI (?9o)^^^ (C - ^o) , n <T <1, 

where r* G (0,1) and measnrable space (C [r*, 1] , f8) of continnons on [r*, 1] 
fnnctions. Here 03 is the corresponding borelian a-algebra. Denote by 
hh (r), 0 < r < 1 a d-dimensional standard Wiener process. 

Theorem 1 Suppose that the regularity conditions hold. Then the one-step 
MLE-process "d*, < r < 1 has the following properties: 

1. It is uniformly consistent: for any u > 0 

lim snp ( snp I'd* — do| > ^ ) =0. (10) 

2. For any r* G (0,1) the random process r}r,T (do) , t* < r < 1 converges 
in distribution in (C [r*, 1] , 03) to the vector-process IT (r), r* < r < 1. 

3. It is asymptotically efficient in the sense ([3]) for any p > 0. 

Note that for a fixed r we have the asymptotic normality 

Vf{d*-do)^^^{OX (do)-') 


( 11 ) 





Proof. We denote by C the generic constant. The uniform consistency 
ffTIJ]) is verihed as follows 


1?0 


ly 

^ 2 


( sup -l}o\> lA < P^o (\As - ^0 

\ts<t<1 / V 

+ P^J sup >^') 

Vr5<T<l 


< + P^o { sup I A. (4, I > ) 

\ts<t<1 J 




< Cu-PT-P^/^ + P^, I sup ^ 

',r,<r<l At 


r-rT 


S{As,Xt) 


dW 


+ P^o sup 

r5<r<l 


ptT 

Its 


rpS CT (Xt) 

S{As,X,) [SiA,X,)-SiAs,XA 
Ta{xA 




dt 


> 


u 

4C 


We have 

S {{}o,x) - SAts,x) = [ Ao - As)* A^v,x) dv, 

Jo 

where 'dy = iJo + v Ao ~ “drs) uud therefore 

s{As,Xt) [s{do,Xt)-siAs,xA 


sup 

r5<r<l 


prT 

Its 

rT 


< 


< 


T a {XA 

s{As,Xt) \s{^o,Xt)-s{As,Xt) 


dt 


IT^ 


1 rT 



Ta{XtY 

s{As,Xt) s{dy,xY 


Ta{XT 


'0 JT^ 

< ^ [^ il + \XA)dt 


f tS 


n 


■dt 

dtdn I'do — 

-^0, 


( 12 ) 


where we used the condition S {d, x), a {x)~^ G V and the consistency of 
dr^. Recall that by condition Aq (0) the invariant density has all polynomial 
moments and therefore we obtain the convergence to zero of all moments for 
these normalized integrals. 
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Further, we have for any A G and m > 0 


f-rT 


P^o ( sup 

TS<T<1 VT 


< ( sup 

TS<T<1 




dW, 


rpS a (Xf) 

j'S 


> 


vT^I‘^ 




dW, 


> 





X*S{^rs,X,) 


rr^+l 
Pi 2 

AC 

2m 


dhE 






df 


< 


C 


jy2m'jpm5 


0 , 


where we used the Burkholder-Davis-Gundy (BDG) inequality (see, e.g., [5], 
Theorem 3.28). Therefore the consistency ([H]) is proved. 

To prove the weak convergence of ('do) ,t* < "t < 1 we write the 
representation 

rjr,T (do) = Vr,T (^o) + P (t, X^^^) + R (t, X(^P) , 


where 


Vr,T (^?o) = I ('do) 


1/2 1 S (^drps, Xt) 

y/r Jt^ CT (Xt) 


dVG. 


Then we verify that 

1. We have the uniform convergence 

sup |P (r, I ^ 0, sup |P (r, I —)■ 0. (13) 

Tt<r<l 


2. We have the convergence of hnite-dimensional distributions: for any 

k = 1,2,... and r* < Ti < ... < < 1 

(hri,r (^?o) , • • • , ??rfc,r ( 4 )) ^ (W(Ti),...,W (Tk)^ . (14) 

3. There exists a constant G > 0 such that 

sup E,Jo ('do) - hr2,T (^?o)|^ <C\Ti- T2f . (15) 

j9o6A 
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We consider T > where T* = r^f \ i.e., ts <t^. We can write 

rVfl - 4) = rVfl (4 - do) 




-1 1 


frT 


S{As,Xt) 


Vt 


dWi 




-1 1 


IrpS <7 (Xt) 

s {As,Xt) [s{A,Xt)-s{As,Xt)] 


Vf 


dt 


= I(do)"'/' 


Vt 


TS (J (Xt) 

(r, X^'^) + R (t, , 

rpS a [At) 


(*tT c 


where we denoted Un = tVTI (do)^'^^ (d^-^ — do) and 


P{t,X^A = ^ 


1(4 


tT 


frT 


f 'j'S 


g(i)o,.Yi)S(tfo,X,)- 

a(X,) 


dt 


Here R (r, is a difference of the corresponding expressions with d^-^ 

and do- 

Then we remark that by the central limit theorem for ordinary integrals 
(see e.g., ESI or m, Proposition 1.23) we have 

XP {pX^V X* ^Af{0,D^) 

with some finite limit variance D^. 

Hence 

u^^T-T^P (r, X^V = (do)'/' {As - A) P {t, X^A ^ 0 

because (d^-^ — do) and P (r, X’"^) are bounded in probability. To verify 
that this convergence is uniform in r G [p , 1] we give some details. Let us 
denote 


/i(do,T) = A I (do) - 


S (do,x) S {A, A* 


a A) 

H (do, x) = I h (do, y) f (A, y) dy. 


X* 


Here A G |A| = 1. Then E^(,/i(do,4) = 0 and by Ito formula we can 
write 


XP (r, X^A 



1 


tVT 


tT 

hAo,Xt) dt 



2HAo,Xt) 

J, aiXt)fiA,Xt) 


dWf 


h (do, x) dx, 
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Further by BDG inequality: for any > 0 and m > 0 


f 


i9o sup 


Vt*<t<1 t^/T 


i-tT 


/o a{X,)f{^o,X,) 


dW, 


> pT'i 


rrT 


< K^.(un)-^^T-^^E 


1?0 




dH{^o,Xtf 

lo a{X,ff{A,Xty 


dt 


< Cu-^^T 


— 2mrp—mS 


0 . 


The existence of the related moments is verified following the same steps as 
in HU, p.31-33. To show that R (r, —>■ 0 we verify several estimates 

like 


E, 


)9o 


rrT 


S{^o,Xt) 


rrT 


Vt 

= E 


Irs a{xy Vt 




1?0 


E, 


1?0 


T'5 ct (Xt) 

s{^o,xy-s{d^„Xt) 


dW, 


VT 


' j'S 




iW, 


XJ'S 


r*rT 

< V I E 

I J'S 

rrT 


•do 


s(d„,x,)-s(d,„x,) 


<y [X,) 


dt 


T 

I E,. (1 + IJf.r) 14 -A\" dt^o (T-‘) ^ 0 (16) 


/ 'j'S 


and (below dv = 'do + v — 'do)) 


r-T^^( 4 ,Xt) S{do,Xt)-S(Xs,Xt 


VT 


criXtY 

^ s{V„xYs{x,xV 

T Jo Jo CT{XtY 


dt 


df dn 


< C 


Ti (4 - A) 


1 rT‘ 





0 JO 


s{V„Xt)s{x,Xty 

^ix,Y 


dfdn 


T-— 


= O T-- 


0 . 


( 17 ) 


Note that the convergence flTB]) is uniform w.r.t. r G [r*, 1] because by BDG 
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inequality: for any u > 0 and any m > 0 


Pl?0 


sup 

r«<r<l 


1 

vf 


r-rT 


/ 'jpS 


cr{Xt) 


dW, 



< 


K„ 





S(A,X,)-SiA„X,) 

a{X,) 


2m 

df —y 0 


with some constant K^, > 0. 

The convergence 0121) follows from the central limit theorem for the vector- 
stochastic integrals (see, e.g., m], Proposition 1.21). Further for Ti < T 2 we 
have 


E^o l^ri.T ('do) - hT2,r (do)l^ - ^E,9g 


72 ~ 'Ti ^ 
< —^ 


nT 2 T c 


ItiT 


S{^o,Xt) 


dWt 


T 


rT2T 

^(do,X*) 

JtiT 

(X.) 


dt < C Iti — T2\ 


where the constant C > 0 does not depend on -do and T. 

More detailed analysis based on the same estimates shows that we have 
the convergence of moments uniform on compacts 


sup 

)9o6A 


x/^I(do)'/" (d*-do)"^E|C| 


From this convergence and the continuity of the matrix I (d) follows the 
asymptotic efficiency ([3]) of the one-step MLE. 


3.2 Two-step MLE {6 G (|, |]) 

The learning time interval can be shorter. Let us take the first estimator 
■drs constructed by the observations X'^ = , 0 < f < T^) with S G (|, |]- 

We suppose that this estimator is consistent, asymptotically normal and the 
moments converge too: 

Vrs = i^Ts - do) ^ AA (0, M (do)), sup \vrgf < C, 

for any p > 0. Here M (do) is some matrix and C > 0 does not depend on 
T. As before it can be the MLE, MDE, BE or the EMM. 

Introduce the second preliminary estimator, which is estimator-process 

d, = d., + (rT)-'/'! , r e [r^, 1], (18) 
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where 


l^rT (tf, X-J) = ^ r |dA-, - S (tf, X,) dt] . (19) 

VtT Jts a [Xt) 

The two-step MLE-process we define as follows 

= A + Kt , rs<T<l, (20) 

where 

A,t (di, dj, x-J) = r [dX, - S (dj, X,) d(|. 

Note that K^t {&■, "&■, X^J) = A^t ("d, X^J). 

Theorem 2 Suppose that the eonditions of regularity hold. Then the two- 
step MLE-process 'd**,Ts < r < 1 is uniformly consistent, asymptotically 
normal 


Vt «* -do)^Af (0, r-'l (4)-') , 

and asymptotically efficient. The random process 

t1t,t {'do) = t^/TI (K* - ’do), r* < r < 1 


for any r* G (0,1) converges in distribution to the d-dimensional standard 
ITzener process Vh (r), r* < r < 1. 


Proof. The proof will be given in two steps. First we show that the 
estimator-process -dr is such that 


sup 

i^oSA 


Ti {dr - do) 


< C 


with 7 G (|, 1) and then we can use the proof of the Theorem [H where the 
mentioned properties are already established. 
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Let us take such 7 > 4 that 7 < 26 . We have 


Ti - ^ 0 ) = [As - a) + 


1 ( 4 ) ' rrTS(d^„X,'^ 


-Vt 


/rp5 


(j{Xt 


dWTV 


T2 


-1 


rTS{As,Xt 


tT Jts a {XtY 


S{A,xY-S{As,Xt 


dt 




I 7— 1 

O tV 


HAs)- 



1 rrTS{As,Xt]SiA:Xt) 


0 JT'5 


tT a {XtY 


-dtdu 


Here Vrg = T 2 ( — do I . We can write 


T— 


Kd.J - 



1 


= Vf 


I(d, 


/o Jt^ tT a (Xt) 

s (A, XYS (A, XY 


-dt du 


-dt 


'0 


_ 1—7+5 

T 2 


+ 


IK)-I (do) 


TTa(XY^ 

, A' S(A,XYS(A,XY*,^2^ 


T— + 


rTa{XA 


-dt T— 


1 rrT s (do, XY s (do, XY* -s{As,Xt)siA, XY 



0 JT^ 

. 1 + 7 — 


tT a {XY^ 


-dt duT 2 


= o (r--^ ] + o (1 + o (r-‘+^ ] + o(t 


( 21 ) 


Recall that the components of the vector S (d, x), of the matrix § (d, x) and of 
the function a (x)~^ have polynomial majorants and the invariant density has 
exponentially decreasing tails. Therefore it can be shown that the moments 
converge too. Moreover for any p > 0 

sup E,,, 

i9o6A 
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We have the similar relations for the two-step MLE-process too. Indeed 


^ frTS[A„X, 

Vf {’Sr --Do) = Vf (A - A) + / \ y / div. 




1(^0”' r^S\^rs,Xt 


rVf Jt^ a {Xtf 

= <i (A) 


.-1 


rl rrT SU^,,XASi^,,Xtr 

1(A)- / ^- - -5-dtdn 

^ ^ Jo Jts rTa(xA 


■dW,. 


1{A)~" r^Si^As,Xt 


t\/T Jt^ O' (Xt) 
where v* = Ti — -do) . Then the corresponding relations are 

.1 i.rTS(^As,Xt)S{A,XA 

Jo Jt^ 


Id, 


'T5 


rTa{XA 


-dt dn 


1 — 7 

T— 


= Vt 


Jo tT a (Xt) 


7 

T-2 


. [I(.7 - n^„)] TV + r rv 

Jo tT a (Xt) 


1 rrT s (do, Xt) S (do, XtY - S (As,XAS {A,Xt) 



0 JT^ 


tT a (XtY 


-dt dnT 2^ 


= O (T-2) + O f T-^") + o(+ o(T-^ 


( 22 ) 


Therefore 


T 


At (A* - do) = ^AAi^aWt + o (i) 


AT Jo (o{Xt) 

^AA(0,rI(do)-'). 

The weak convergence rj^-^T ('do ), "T. ^ < 1 now follows from the proof 

of the Theorem [T] 


3.3 Example 

Snppose that the observed process is 

dXi = -(Xt- d)^ dt + dWt, Xo, 0 < t < T 
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where '(9 G 0 = (a, fe). It is easy to see that the conditions of regularity are 
fulhlled and the process is ergodic with the density of invariant distribution 


81/4 

/ x) = exp 


{x — dY 


= (f (x — 19). 


Note that the MLE of the parameter i9 can not be written in explicit form. 
Let us take (^ = |- We have for the empirical mean (estimator of the method 
of moments) the consistency 


i 9 j^ 3/4 — 


1 


^ 7 ’ 3/4 


T3/4 


Xt df —)■ — do 


and asymptotic normality 


^7'3/4 


Ts (d 3 - do) = ;^ / (W - do) dt^Af (0, , 

Ts Jo 


where 

= 4E 


1?0 


(y-^o)f(A,y) 

/( 4,0 


dy = 4 Eo 


r^o 


y^ {y) 

1-00 (^o) 


dy 


Here the random variable ^o has the density function (p{x). The Fisher 
information I does not depend on d and the one-step MLE-process is 


rrT 


d* = i9x3/4 — 


-tVi. 


’ 'j'S 


{Xt - dr3/4)^ dXt + {Xt - d-r3/4)^ dt 


This estimator by Theorem [T] is uniformly consistent, asymptotically normal 

^/7T (d;-do) (0,1-1) 

and asymptotically efficient. 

If the learning interval is then the preliminary estimator -drps/s 

has the rate of convergence We take the second estimator-process as 


dr = drp3/3 + 


ItT 


f-rT 


IX 3 /S 


Xg — l9rp3/8 


dXs -|- (— i9rp3/8\ ds 


For this estimator the relation 

E,,JtW 6 (d,-do)f ^0 
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holds. Therefore by Theorem [2] the two-step MLE-process 



2 


dX, + {X,-drfds 


is asymptotically normal 


( 0 , 1 - 1 ) ^ 


The similar estimator-processes can be constructed and in the case of 
two-dimensional parameter = [a, j3) and the observations 

dXt = -13 {Xt - af dt + dWt, Xq, 0<t<T 


where (3 > 0. Indeed suppose that h = |. the preliminary estimator drp3/4 
[dj<3/4,, l3rp3/4) can be 



The invariant density is 



and the Fisher matrix I {/3) is diagonal. Therefore the one and two-step 
MLE-processes can easily by written. 

3.4 Discussions 

Note that the process of construction of multi-step estimators can be contin¬ 
ued. For example, if the initial rate is with 6 G (|, |], then we can use 
once more one-step device to improve the rate of preliminary estimator up 
to 7 G (|, |), where 7 satishes the condition 7 < 2h and so on. Therefore 
the asymptotically efficient estimator-process will be three-step MLE. 

We used two estimators drg and dr because the estimator dr depends on 
the whole trajectory and the stochastic integral 
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is not well defined. Another possibility to avoid this problem is to replace 
the stochastic integral by ordinary integrals and to nse one estimator only as 
follows. 

Snppose that the fnnctions S {'d, x ), S (-d, x) and a {x) are continnously 
differentiable w.r.t. x and the derivatives belong to the class V. Introdnce 
the vector-process 




S (d, X,) a (A,) a' (A^) - ^ X,) S A,) 

Its V^a{X,f 


We have (by the Ito formnla) 


(^?,A^r) =ArT (^9,A^T) . 


The two-step MLE-process in this case is 

+ AjT) , TS<T<1. 


It can be shown that this estimator is asymptotically eqnivalent to i?** and 
has the same asymptotic properties as those described in the Theorem [2j 
The calcnlation of the Fisher information matrix for some models can 
be a difficnlt problem. In snch cases we can replace the Fisher information 
matrix I (-d) by its empirical version 




1 p s{^,Xs)S{^,XsY 
tJo <7 {x,Y 


The proposed in this work construction can be easily generalized to many 
other statistical models. At particularly, it “works” in the case of small noise 
asymptotic 


dXt = S{d,t,Xt) dt + ea{t,Xt) dWt, Aq = xq, 0 < f < T, 


where T is fixed and the asymptotics corresponds to e 0. We introduce a 
learning time interval [0, t^], where —?• 0 and for some values of d > 0 

we show that the one-step MLF-process 




[dX,-S{A,,s,XY] , 


t e [A,r] 
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is asymptotically efficient estimator for all t G [r*,T] with any 0 < r* < T 
|13] . Note that for this model the constrnction of the consistent preliminary 
estimator of d-dimensional parameter "d is possible if the observed process 
Xt is k dimensional and k > d. 

This mnlti-step ML E-pro cesses can be realized and in the case of estima¬ 
tion of parameter d by the discrete time observations X"’ = (Xij,..., X ^^), 
tj = of the diffusion process 

dXt = S{t,Xt) dt + a{^,t,Xt) dWt, Xo = xo, 0<t<T. 

Here we suppose that the time of observation T is hxed and n ^ oo. The 
corresponding multi-step pseudo MLE-process is asymptotically efficient pQ. 

For the nonlinear autoregresive model 

Xj+i = S (^9, Xj) + Ej+i, j = 0,l,...,n-l 

the similar multi-step MLE-process provides asymptotically efficient estima¬ 
tor process too m. 

The construction of the multi-step MLE-processes can be done in the case 
of inhomogeneous Poisson processes, i.i.d. observations and so on. 

In the work [T3] we apply the one-step MLE in the construction of the 
goodness-of-ht tests based on score-function-processes. 

Note as well that the one-step MLE-process device allowed us to construct 
asymptotically efficient estimator of the paramezters of hidden telegraph sig¬ 
nal [8]. 
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